A Redesign of the ACGME Resident/Fellow Survey Through Expert Reviews and Cognitive Interviews.

Background: Since 2003, the Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents and fellows in its accredited programs. The Resident/Fellow Survey is a screening and compliance tool that programs can also use for continuous quality improvement. However, stakeholders have reported potential problems with the survey's overall quality and credibility. Objective: To redesign the 2006 Resident/Fellow Survey using expert reviews and cognitive interviews. Methods: In 2018-2019, the ACGME redesigned the Resident/Fellow Survey using an iterative validation process: expert reviews (evidence based on content) and cognitive interviews (evidence based on response processes). Expert reviews were conducted by a survey design firm and an ACGME Task Force; cognitive interviews were conducted with a diverse set of 27 residents and fellows. Results: Expert reviews resulted in 21 new survey items to address the ACGME's updated accreditation requirements; these reviews also led to improvements that align the survey items with evidence-informed standards. After these changes were made, cognitive interviews resulted in additional revisions to sensitive items, item order, and response option order, labels, and conceptual spacing. In all, cognitive interviews led to 11 item deletions and other improvements designed to reduce response error. Conclusions: Expert reviews and cognitive interviews were used to redesign the Resident/Fellow Survey. The content of the redesigned survey aligns with the updated accreditation requirements and items are written in accordance with evidence-informed standards. Additionally, cognitive interviews resulted in revisions to the survey that seem to improve respondents' comprehension and willingness to respond to individual survey items.

Beyond Training the Next Generation of Physicians: The Unmeasured Value Added by Residents to Teaching Hospitals and Communities.

Following medical school, most newly graduated physicians enter residency training. This period of graduate medical education (GME) is critical to creating a physician workforce with the specialized skills needed to care for the population. Completing GME training is also a requirement for obtaining medical licensure in all 50 states. Yet, crucial federal and state funding for GME is capped, creating a bottleneck in training an adequate physician workforce to meet future patient care needs. Thus, additional GME funding is needed to train more physicians. When considering this additional GME funding, it is imperative to take into account not only the future physician workforce but also the value added by residents to teaching hospitals and communities during their training. Residents positively affect patient care and health care delivery, providing intrinsic and often unmeasured value to patients, the hospital, the local community, the research enterprise, and undergraduate medical education. This added value is often overlooked in decisions regarding GME funding allocation. In this article, the authors underscore the value provided by residents to their training institutions and communities, with a focus on current and recent events, including the global COVID-19 pandemic and teaching hospital closures.

Checking in on check-out: Survey of Learning Priorities in Primary Care Residency Teaching Clinics.

Background: Despite focus on increasing the quality of ambulatory education training, few studies have examined residents' perceptions of learning during case discussions with their preceptors (i.e., "check-out"). The objective of this study was to assess the difference between residents' and preceptors' perceptions of behaviors that should occur during check-out discussions. Methods: We conducted a cross-sectional survey of categorical internal medicine and family medicine residents and preceptors. The survey was distributed electronically and assessed 20 components of the check-out discussion. Results: Of 38 preceptors, 22 (61%) completed the survey. Of 172 residents, 82 (48%) completed the survey. For residents, we identified discrepancies in desired and perceived check-out behaviors. Specifically, utilizing a dependent sample t-test, residents felt that all 20 areas needed additional teaching during check-out (P < 0.05). Preceptors believed that demonstrating physical examination skills in the patient room during check-out was significantly more important than did residents (P = 0.01). Increasing years of preceptor experience did not statistically relate to their valuation of components important to residents. Discussion: Our research highlighted a major deficiency in training in the check-out process, with residents desiring more patient management education in all components. Moreover, faculty and residents do not necessarily agree with what is an important focus in the "teachable moment." Our results serve as a training needs assessment for future faculty development seminars and highlight the need to consider resident learning needs in general.

A Team Disclosure of Error Educational Activity: Objective Outcomes.

Medical errors can involve multiple team members. Few curricula are being developed to provide instruction on disclosing medical errors that include simulation training with interprofessional team disclosure. To explore more objective evidence for the value of an educational activity on team disclosure of errors, faculty developed and assessed the effectiveness of a multimodal educational activity for learning team-based disclosure of a medical error. This study employed a methodological triangulation research design. Participants (N = 458) included students enrolled in academic programs at three separate institutions. The activity allowed students to practice team communication while: (1) discussing a medical error within the team; (2) planning for the disclosure of the error; and (3) conducting the disclosure. Faculty assessed individual student's change in knowledge and, using a rubric, rated the performance of the student teams during a simulation with a standardized family member (SFM). Students had a high level of preexisting knowledge and demonstrated the greatest knowledge gains in questions regarding the approach to disclosure (P < .001) and timing of an apology (P < .001). Both SFMs and individual students rated the team error disclosure behavior highly (rho = 0.54; P < .001). Most participants (more than 80%) felt the activity was worth their time and that they were more comfortable with disclosing a medical error as a result of having completed the activity. This activity for interprofessional simulation of team-based disclosure of a medical error was effective for teaching students about and how to perform this type of important disclosure.

Beyond the United States Medical Licensing Examination Score: Assessing Competence for Entering Residency.

Assessments of physician learners during the transition from undergraduate to graduate medical education generate information that may inform their learning and improvement needs, determine readiness to move along the medical education continuum, and predict success in their residency programs. To achieve a constructive transition for the learner, residency program, and patients, high-quality assessments should provide meaningful information regarding applicant characteristics, academic achievement, and competence that lead to a suitable match between the learner and the residency program's culture and focus.The authors discuss alternative assessment models that may correlate with resident physician clinical performance and patient care outcomes. Currently, passing the United States Medical Licensing Examination Step examinations provides one element of reliable assessment data that could inform judgments about a learner's likelihood for success in residency. Yet, learner capabilities in areas beyond those traditionally valued in future physicians, such as life experiences, community engagement, language skills, and leadership attributes, are not afforded the same level of influence when candidate selections are made.While promising new methods of screening and assessment-such as objective structured clinical examinations, holistic assessments, and competency-based assessments-have attracted increased attention in the medical education community, currently they may be expensive, be less psychometrically sound, lack a national comparison group, or be complicated to administer. Future research and experimentation are needed to establish measures that can best meet the needs of programs, faculty, staff, students, and, more importantly, patients.

Effects of gender in resident evaluations and certifying examination pass rates.

BACKGROUND: Though the proportion of female Internal Medicine (IM) residents and faculty has increased, there is minimal large scale modern data comparing resident performance by gender. This study sought to examine the effects of resident and faculty gender on resident evaluations. METHODS: Retrospective observational study over 5 years in a single IM program. IM certifying examination pass rates were obtained from the American Board of IM. RESULTS: Four hundred eighty-eight residents (195 women, 293 men), evaluated by 430 attending physicians (163 women, 270 men) were included. Twelve thousand six hundred eighty-one evaluations between 2007 and 2012 were analyzed. Female residents scored higher in two domains (Medical Interviewing, and Interpersonal and Communication Skills) (p < 0.01 for each), with no significant difference between genders for the other domains (Medical Knowledge, Overall Patient Care, Physical Examination, Procedural Skills, Professionalism, Practice Based Learning and Improvement, System Based Practices and Overall score). There were no differences in scoring between female and male attending physicians. There were no differences in certifying examination scores between women and men among graduating residents. National pass rates for women were not statistically different to pass rates for men from 1987 to 2015. CONCLUSIONS: Data from one large academic medical center demonstrate higher ratings for female residents on performance domains reflecting bedside care and interpersonal skills, with similar scores for medical knowledge and remaining domains. No significant difference was seen locally in certifying examination scores, nor in recent national pass rates, an objective measure of medical knowledge. Despite imbalanced female representation in areas of medicine, our data suggest that gender-based disparities in Internal Medicine resident medical knowledge and physician competency are no longer present.

Improving Performance on Preventive Health Quality Measures Using Clinical Decision Support to Capture Care Done Elsewhere and Patient Exceptions.

Preventive services required for performance measurement often are completed in outside health systems and not captured in electronic medical records (EMRs). A before-after study was conducted to examine the ability of clinical decision support (CDS) to improve performance on preventive quality measures, capture clinician-reported services completed elsewhere, and patient/medical exceptions and to describe their impact on quality measurement. CDS improved performance on colorectal cancer screening, osteoporosis screening, and pneumococcal vaccination measures ( P < .05) but not breast or cervical cancer screening. CDS captured clinician-reported services completed elsewhere (2% to 10%) and patient/medical exceptions (<3%). Compared to measures using only within-system data, including services completed elsewhere in the numerator improved performance: pneumococcal vaccine (73% vs 82%); breast (69% vs 75%), colorectal (58% vs 70%), and cervical cancer (53% vs 62%); and osteoporosis (72% vs 75%) screening ( P < .05). Visit-based CDS can capture clinician-reported preventive services, and accounting for services completed elsewhere improves performance on quality measures.

The Next Accreditation System.

The Accreditation Council for Graduate Medical Education has implemented a new accreditation system for graduate medical education in the United States. This system, called the Next Accreditation System, focuses on more continuous monitoring of the outcomes of residency training, and for high-quality programs, less on the detailed processes of that training. This allows programs to innovate to best meet the needs of their trainees and communities. This new system also reviews the clinical learning environment at each institution sponsoring graduate medical education, focusing on professionalism, trainee supervision, duty hour and fatigue management, care transitions, and integration of residents into patient safety and health care quality. This Next Accreditation System is too new to fully assess its outcomes in better preparing residents for medical practice. Assessments of its early implementation, however, suggest we can expect such outcomes in the near future.

Implementation of competency-based medical education: are we addressing the concerns and challenges?

CONTEXT: Competency-based medical education (CBME) has emerged as a core strategy to educate and assess the next generation of physicians. Advantages of CBME include: a focus on outcomes and learner achievement; requirements for multifaceted assessment that embraces formative and summative approaches; support of a flexible, time-independent trajectory through the curriculum; and increased accountability to stakeholders with a shared set of expectations and a common language for education, assessment and regulation. OBJECTIVES: Despite the advantages of CBME, numerous concerns and challenges to the implementation of CBME frameworks have been described, including: increased administrative requirements; the need for faculty development; the lack of models for flexible curricula, and inconsistencies in terms and definitions. Additionally, there are concerns about reductionist approaches to assessment in CBME, lack of good assessments for some competencies, and whether CBME frameworks include domains of current importance. This study will outline these issues and discuss the responses of the medical education community. METHODS: The concerns and challenges expressed are primarily categorised as: (i) those related to practical, administrative and logistical challenges in implementing CBME frameworks, and (ii) those with more conceptual or theoretical bases. The responses of the education community to these issues are then summarised. CONCLUSIONS: The education community has begun to address the challenges involved in implementing CBME. Models and guidance exist to inform implementation strategies across the continuum of education, and focus on the more efficient use of resources and technology, and the use of milestones and entrustable professional activities-based frameworks. Inconsistencies in CBME definitions and frameworks remain a significant obstacle. Evolution in assessment approaches from in vitro task-based methods to in vivo integrated approaches is responsive to many of the theoretical and conceptual concerns about CBME, but much work remains to be done to bring rigour and quality to work-based assessment.

What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-of-training entrustable professional activities for geriatric medicine.

Entrustable professional activities (EPAs) describe the core work that constitutes a discipline's specific expertise and provide the framework for faculty to perform meaningful assessment of geriatric fellows. This article describes the collaborative process of developing the end-of-training American Geriatrics Society (AGS) and Association of Directors of Geriatric Academic Programs (ADGAP) EPAs for Geriatric Medicine (AGS/ADGAP EPAs). The geriatrics EPAs describes a geriatrician's fundamental expertise and how geriatricians differ from general internists and family practitioners who care for older adults.

The good, the bad and the early adopters: providers' attitudes about a common, commercial EHR.

RATIONALE, AIMS AND OBJECTIVES: To describe primary care providers' (PCP) attitudes about the impact of a mature, commercial electronic health records (EHR) on clinical practice in settings with experience using the system and to evaluate whether a provider's propensity to adopt new technologies is associated with more favourable perceptions. METHOD: We surveyed PCPs in 11 practices affiliated with three health systems in Texas. Most practices had greater than 5 years of experience with the Epic EHR. The effect of early adopter of technology status was evaluated using logistic regression. RESULTS: One hundred forty-six PCPs responded (70%). Most thought the EHR had a positive impact on routine tasks, such as prescription refills (94%), whereas fewer agreed for complex tasks, such as delivery of guideline-concordant care for chronic illnesses (51%). Two-thirds (62%) thought it interfered with eye contact with patients, and 40% reported that it interfered with in-visit communication. Early adopters of technology reported greater positive effects of the EHR, even after adjusting for age, ranging from 2% to 15% higher on satisfaction ratings. CONCLUSION: PCPs practicing in settings with considerable experience using a common commercial EHR identified many positive effects, as well as two key areas for improvement - patient centredness and intelligent decision support. Providers with a propensity to adopt new technologies have more favourable perceptions of the EHR.

The Power of Collaboration: Experiences From the Educational Innovations Project and Implications for the Next Accreditation System.

BACKGROUND: The Internal Medicine Educational Innovations Project (EIP) is a 10-year pilot project for innovating in accreditation, which involves annual reporting of information and less-restrictive requirements for a group of high-performing programs. The EIP program directors' experiences offer insight into the benefits and challenges of innovative approaches to accreditation as the Accreditation Council for Graduate Medical Education transitions to the Next Accreditation System. OBJECTIVE: We assessed participating program directors' perceptions of the EIP at the midpoint of the project's 10-year life span. METHODS: We conducted telephone interviews with 15 of 18 current EIP programs (83% response rate) using a 19-item, open-ended, structured survey. Emerging themes were identified with content analysis. RESULTS: Respondents identified a number of the benefits from the EIP, most prominent among them, collaboration between programs (87%, 13 of 15) and culture change around quality improvement (47%, 7 of 15). The greatest benefit for residents was training in quality improvement methods (53%, 8 of 15), enhancing those residents' ability to become change agents in their future careers. Although the requirement for annual data reporting was identified by 60% (9 of 15) of program directors as the biggest challenge, respondents also considered it an important element for achieving progress on innovations. Program directors unanimously reported their ability to sustain innovation projects beyond the 10-year participation in EIP. CONCLUSIONS: The work of EIP was not viewed as "more work," but as "different work," which created a new mindset of continuous quality improvement in residency training. Lessons learned offer insight into the value of collaboration and opportunities to use accreditation to foster innovation.

Use and satisfaction with key functions of a common commercial electronic health record: a survey of primary care providers.

BACKGROUND: Despite considerable financial incentives for adoption, there is little evidence available about providers' use and satisfaction with key functions of electronic health records (EHRs) that meet "meaningful use" criteria. METHODS: We surveyed primary care providers (PCPs) in 11 general internal medicine and family medicine practices affiliated with 3 health systems in Texas about their use and satisfaction with performing common tasks (documentation, medication prescribing, preventive services, problem list) in the Epic EHR, a common commercial system. Most practices had greater than 5 years of experience with the Epic EHR. We used multivariate logistic regression to model predictors of being a structured documenter, defined as using electronic templates or prepopulated dot phrases to document at least two of the three note sections (history, physical, assessment and plan). RESULTS: 146 PCPs responded (70%). The majority used free text to document the history (51%) and assessment and plan (54%) and electronic templates to document the physical exam (57%). Half of PCPs were structured documenters (55%) with family medicine specialty (adjusted OR 3.3, 95% CI, 1.4-7.8) and years since graduation (nonlinear relationship with youngest and oldest having lowest probabilities) being significant predictors. Nearly half (43%) reported spending at least one extra hour beyond each scheduled half-day clinic completing EHR documentation. Three-quarters were satisfied with documenting completion of pneumococcal vaccinations and half were satisfied with documenting cancer screening (57% for breast, 45% for colorectal, and 46% for cervical). Fewer were satisfied with reminders for overdue pneumococcal vaccination (48%) and cancer screening (38% for breast, 37% for colorectal, and 31% for cervical). While most believed the problem list was helpful (70%) and kept an up-to-date list for their patients (68%), half thought they were unreliable and inaccurate (51%). CONCLUSIONS: Dissatisfaction with and suboptimal use of key functions of the EHR may mitigate the potential for EHR use to improve preventive health and chronic disease management. Future work should optimize use of key functions and improve providers' time efficiency.